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PREFACE 


This  documented  briefing  describes  the  results  of  an  inquiry  conducted  by 
RAND’s  Science  &  Technology  Policy  Institute  for  the  Office  of  Science  and 
Technology  Policy  (OSTP)  of  the  Executive  Office  of  the  President  of  the 
United  States.  OSTP  asked  RAND  to  provide  insights  into  improving  the 
efficiency  and  effectiveness  of  government-sponsored  international 
collaboration  in  science  and  technology.  This  document  can  be  used  as  the 
basis  for  a  workshop  addressing  the  questions  of  creating  effective 
international  linkages  in  science  and  technology. 

This  project  had  three  goals:  (1)  to  improve  understanding  of  the  dynamics  of 
international  collaboration  in  science  and  technology,  (2)  to  provide  tools  for 
policymakers  seeking  to  improve  the  effectiveness  and  efficiency  of 
collaboration,  and  (3)  to  coordinate  with  analysts  conducting  similar  studies  in 
different  countries.  Four  case  studies  conducted  for  the  RAND  effort  provide 
the  research  from  which  we  draw  lessons  learned  about  linking  effectively. 
Policymakers  faced  with  decisions  about  participation  and  resource 
commitment  may  find  helpful  ideas  about  forming  and  supporting 
collaboration. 

The  motivation  and  methodology  for  this  study  derives  from  a  broad  set  of 
consultations  among  analysts  in  a  number  of  countries,  including  Canada, 
Japan,  Korea,  and  countries  of  the  European  Union  (EU).  Each  of  these 
countries  and  the  EU  is  fielding  a  team  of  researchers  conducting  a  parallel 
study  on  the  same  four  cases.  Once  all  the  different  country  studies  are 
complete,  a  workshop  will  be  conducted  and  a  final  coordinated  report  will  be 
compiled  to  examine  collaboration  from  a  number  of  different  national 
perspectives.  It  is  hoped  that  the  lessons  learned  from  the  individual  country 


studies,  and  those  derived  from  comparing  the  studies,  will  provide 
government  policymakers  with  a  tool  kit  of  policy  options  to  help  in  thinking 
more  strategically,  creatively,  and  efficiently  about  advancing  science  and 
technology. 

This  project  has  the  advantage  of  both  studying  collaboration  as  well  as  being 
a  collaboration.  We  hope  to  be  able  to  learn  as  much  from  our  own  efforts  at 
conducting  an  international  collaborative  project  as  we  do  from  studying 
successful  collaborations.  These  lessons  will  be  shared  and  enumerated  at  a 
workshop  in  2003. 

The  examination  of  successful  collaboration  cases  from  the  United  States’ 
perspective  is  complete  with  the  publication  of  this  documented  briefing. 
However,  the  view  from  the  United  States  is  only  one  small  part  of  a  larger 
picture.  Without  understanding  how  other  countries  view  the  same  programs, 
how  well  these  organizational  structures  worked  for  other  countries,  and  what 
issues  and  problems  they  faced,  this  document  would  only  be  of  limited  use. 
We  welcome  feedback  from  any  reader  or  reviewer.  Nevertheless,  this  is  an 
interim  product — we  await  the  results  from  other  analysts  in  other  countries 
before  we  can  tell  the  full  story  of  lessons  learned  from  participation  in 
international  scientific  and  technical  collaborations. 

Created  by  the  U.S.  Congress  in  1991  as  the  Critical  Technologies  Institute,  it 
was  renamed  the  Science  and  Technology  Policy  Institute  in  1998.  The 
Institute  is  a  federally  funded  research  and  development  center  sponsored  by 
the  National  Science  Foundation  and  managed  by  RAND.  The  Institute’s 
mission  is  to  help  improve  public  policy  by  conducting  objective,  independent 
research  and  analysis  on  policy  issues  that  involve  science  and  technology.  To 
this  end,  the  Institute 

•  supports  OSTP  and  other  executive  branch  agencies,  offices,  and  councils 
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•  helps  science  and  technology  policymakers  understand  the  likely 
consequences  of  their  decisions  and  choose  among  alternative  policies 

•  improves  understanding  in  both  the  public  and  private  sectors  of  the  ways 
in  which  science  and  technology  can  better  serve  national  objectives. 


In  carrying  out  its  mission,  the  Institute  consults  broadly  with  representatives 
from  private  industry,  institutions  of  higher  education,  and  other  nonprofit 
institutions. 


Inquiries  regarding  the  Science  and  Technology  Policy  Institute  or  this 
document  may  be  directed  to: 


Helga  Rippen 

Director,  Science  and  Technology  Policy  Institute 
RAND 

1200  South  Hayes  Street 
Arlington,  Virginia  22202 
Phone:  703-413-1100,  ext.  5574 
Web:  www.rand.org/scitech/stpi 
Email:  stpi@rand.org 
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SUMMARY 


This  briefing  seeks  to  answer  three  questions:  (1)  Why  study  the  subject  of 
formal,  government-sponsored  collaboration?  (2)  What  did  we  learn  from  the 
four  case  studies  that  gave  an  in-depth  look  at  the  U.S.  experience  in 
sponsoring  and  participating  in  these  programs?  and  (3)  Can  these  programs 
be  evaluated  and  are  they  worth  the  extra  effort  that  is  required  to  initiate  and 
sponsor  them? 

This  briefing  is  organized  to  answer  these  questions,  as  well  as  to  raise  points 
of  discussion  and  debate  among  those  interested  in  this  subject.  It  is  presented 
in  a  format  that  draws  lessons  from  the  case  studies  and  then  presents  key 
questions  that  emerged  from  the  cases  that  can  serve  as  a  guide  to  others 
seeking  to  formulate  similar  collaborative  programs. 

The  first  section  discusses  the  growing  role  that  international  collaboration  is 
playing  in  science  and  technology  (S&T).  Here  we  also  discuss  the  case  study 
methodology  used  for  this  study.  The  second  section  presents  a  framework  of 
“lessons  learned”  that  emerged  from  our  examination  of  cases  of  successful 
collaboration.  RAND  created  this  framework  as  a  tool  to  help  policymakers 
create  effective  linkages  in  the  future.  The  third  section  discusses,  from  the 
U.S.  perspective,  what  people  reported  to  us  as  some  of  the  benefits  of 
participating  in  international  collaborations. 

The  briefing  has  two  components:  a  set  of  slides  and  a  written 
accompaniment.  The  two  parts  are  designed  to  be  used  together  and  read  as  a 
report,  and  it  can  also  serve  as  the  basis  for  a  workshop.  Agency  officials 
seeking  to  explore  the  creation  of  formal  international  collaborations  may  wish 
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to  use  this  briefing  to  generate  discussion  and  comment  among  colleagues  and 

potential  collaborators. 

It  is  important  to  set  out  the  context  of  some  common  terms  used  in  this 

report: 

•  S&T  refers  to  the  many  different  investments  made  by  the  governments  in 
basic  research,  in  applied  research,  in  development  of  equipment  and 
standards,  and  in  data  collection  and  analysis  needed  both  to  increase 
knowledge  about  the  natural  world  and  to  help  the  government  in  its 
various  missions. 

•  Research  and  development  (R&D)  is  a  subset  of  S&T  activities.  The  term 
refers  to  programs  and  projects  budgeted  as  “research  and  development” 
by  federal  agencies.  These  are  activities  that  seek  to  apply  the  scientific 
method  to  specific  experimental  questions  identified  by  government 
agencies  as  important  and  validated  by  scientific  peers  as  worthwhile. 

•  Curiosity-driven  research  is  the  set  of  S&T  activities  that  are  proposed  by 
scientists  and  conducted,  usually  as  basic  research,  because  the  subject  is 
not  well  understood  and  where  the  application  of  the  scientific  method  of 
observation  and  experimentation  may  add  to  the  stock  of  knowledge. 

•  Mission-oriented  research  is  the  set  of  S&T  activities  that  are  defined  by 
government  agency  officials  who  commission  or  conduct  research  (usually 
applied  research  or  development)  that  will  advance  knowledge  needed  for 
an  agency  to  carry  out  its  mission. 

•  Policy-oriented  research  is  the  set  of  S&T  activities  that  are  defined  by 
government  officials  or  elected  representatives  to  reach  a  policy-oriented 
goal  using  S&T  as  a  tool. 

•  Cooperation  refers  to  all  the  programs,  projects,  and  support  activities 
sponsored  by  the  U.S.  government  with  foreign  entities  that  have  a 
scientific  or  technical  component.  It  can  include  joint  R&D,  technical 
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assistance,  technology  transfer,  standards  development,  and  other  types  of 
joint  activities. 

Collaboration  refers  to  the  specific  scientific  activities  (research  and 
observation,  experimentation,  data  collection,  publication)  conducted  by 
scientists  working  together  on  a  common  research  project. 

“Champion”  is  a  term  applied  to  a  scientist  who  has  taken  on  the  task  of 
promoting  to  interested  parties — legislators,  other  scientists,  the  public — 
the  value  of  government  funding  for  a  specific  program  or  course  of 
research. 
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EU 

European  Union 

HFSP 

Human  Frontier  Science  Program 

HGP 

Human  Genome  Project 

IMS 

Intelligent  Manufacturing  Systems 

IPCC 

Intergovernmental  Panel  on  Climate  Change 

OSTP 

Office  of  Science  and  Technology  Policy 

R&D 

Research  and  development 

S&T 

Science  and  technology 

This  Briefing  Draws  Larger  Lessons 
from  Four  Programs 

H  SdenaeA  Technoloav  Policy  Institute _ 

Why  Study  this  Subject? 

I.  Formal  Collaboration  is  Growing 

Purpose  for  and  methodology  of  this  study 

What  did  We  Learn  by  Studying  Coiiaboration? 

II.  Dynamic  Research  Needs  Effective  Linkages 

Six  stages  characterize  these  activities 

Key  questions  are  identified  at  each  stage 

Are  these  Programs  Worth  the  Effort? 

III.  New  Ways  are  Needed  to  Measure  Benefits 
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I.  A  GROWING  ROLE  FOR  INTERNATIONAL 
COOPERATION  IN  SCIENCE  AND  TECHNOLOGY 

International  linkages  in  science  and  technology  are  increasing  and  the 
uses  of  science  within  society  are  growing  more  complex.  Co¬ 
authorships  of  scientific  articles  show  a  significant  increase  in 
international  linkages  over  the  past  20  years.  (NSB  2000)  The 
applications  to  which  science  is  put  to  address  global  problems  such  as 
the  climate,  food  supply,  health,  and  economic  growth  are  more 
numerous  and  of  interest  to  more  and  more  countries.  The  growing 
number  of  countries  with  scientific  capacity  has  expanded  the  pool  of 
potential  partners.  (Wagner  et  ah,  2001) 

The  methods  used  by  scientists  to  create  new  knowledge  are  also 
changing.  The  frequency  and  ease  of  travel  increases  the  dynamism  of 
knowledge  sharing.  This  has  led  to  more  robust  networks  of  scientists. 
The  increasing  ubiquity  of  information  and  communications 
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technologies  means  that  scientists  can  share  information  in  real  time. 
This  had  led  to  the  growth  of  distributed  research  in  a  range  of  activities, 
some  are  called  “co-laboratories”  or  “virtual  laboratories,”  and  others 
are  simply  collaborations — where  practitioners  conduct  parallel  research 
in  their  home  labs  and  share  results  in  real  time.  Gibbons  et  al.(1994) 
have  suggested  that  many  areas  of  research  are  growing  more 
multidisciplinary  and  team  oriented. 

In  the  face  of  these  changes,  policymakers  are  faced  with  complex 
choices  about  how  best  to  support  and  participate  in  global  science  and 
technology.  At  intergovernmental  meetings  on  science  and  technology, 
proposals  for  joint  research  are  often  on  the  agenda.  These  are  not 
unwelcome:  When  scientific  goals  are  sound,  governments  can  use  these 
programs  to  enhance  the  productivity  of  national  science,  as  well  as  to 
create  good  will  and  gain  political  capital. 

Governmental  participation  and  support  of  international  science  and 
technology  (S&T)  collaboration  requires  decisions  to  be  made  at  many 
different  points  and  on  a  number  of  levels.  This  project  examined  a 
subset  of  international  scientific  collaborations:  the  formal  government 
programs  that  operate  in  a  distributed  way,  involving  scientists  from  a 
number  of  different  countries  as  well  as  across  different  institutional 
sectors  and  disciplines. 

The  briefing  covers  three  areas:  1)  the  dynamics  of  international 
collaboration  as  a  tool  for  governments;  2)  findings  from  four  case 
studies  we  conducted  on  distributed  international  collaboration;  and  3) 
considerations  of  how  to  measure  the  benefits  of  these  activities,  at  least 
from  the  point  of  view  of  U.S.  policymakers  supporting  this  work. 


2 


Organization  of  Collaboration 

B  Science  &  Technology  Pdicv  I  nsUtute _ 


HFSP=Human  Fromiers  Science  Program;  HGP=Human  Genome  Projeci;  AR=Arclic  Research;  IMS=Inielligenl  Manufacturing  Systems;  IPCC=Iniergovemmenial  Panel  on  Climate 
Change;  CERN=European  Organization  for  Nuclear  Research  ;  ITER  (fusion  research);  ;ISS=International  Space  Station;  AR=Arctic  research;  ODP=Ocean  Drilling  Program; 
CRDF=Cooperative  Research  and  Development  Fund; 
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Why  did  we  focus  on  this  particular  subject?  To  answer  that,  it  helps  to  put 
international  S&T  cooperation  in  context  and  describe  its  dynamics.  The  figure 
on  this  slide  shows  two  axes  that  can  be  counterposed  to  describe  different 
organizational  forms  of  collaboration.  One  axis  runs  from  spontaneous 
(“bottom-up”)  research  deriving  from  the  interests  of  scientists,  to  highly 
organized  research  defined  by  a  funding  party.  These  two  axes  form  four 
quadrants  where  collaborative  research  can  be  characterised.  Activities  on  the 
left  side  of  the  figure  might  be  described  as  “dynamic”  in  that  collaboration 
requires  active  learning  and  sharing  of  tasks,  and  of  information  among 
researchers  who  are  often  geographically  dispersed.  Activities  on  the  right 
might  be  described  as  material/institutional  research  in  that  collaboration  relies 
on  a  shared  resource  or  common  research  location.  Megascience  projects 
could  be  placed  in  the  bottom  right  quadrant:  organized  and  centralized.  The 
projects  that  we  examined  could  be  placed  in  the  bottom-left  quadrant: 
organized  and  distributed  research  activities.  The  differences  in  organization, 
location  of  research,  and  dynamism  of  the  communication,  require  new  ways 
of  managing  that  differ  from  the  practices  that  policymakers  might  use  for 
megascience  or  for  spontaneous  research  at  the  project  level. 
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Formal  Programs  are  a  Tool  for 
International  S&T  Cooperation 

B  SdenosSt  Techndoq^Pdic\/ Institute _ 


Government-sponsored 
international  cooperation 
in  science  and  technology 
-$5.5  billion  in  the  U.S. 


Formal,  government-sponsored 
collaborative  science  and  technology 
programs,  -$80  million 
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Formal  international  collaboration  accounts  for  about  6  percent  of  the 
more  than  $80  billion  the  U.S.  government  spends  on  S&T.  This  set  of 
activities  can  be  characterized  in  several  ways: 

•  The  largest  class  of  activities  is  independent  research  projects 
involving  scientist-to-scientist  linkages  for  those  working  on 
curiosity-driven  projects.  These  projects  could  be  placed  in  the  top 
half  of  the  figure  on  slide  3. 

•  A  second,  but  smaller,  set  of  activities  is  the  formal,  subject- specific 
programs  from  which  the  cases  examined  in  this  study  are  drawn. 
These  are  the  organized-distributed  projects  described  on  slide  3. 

•  A  related  set  of  formal,  equipment-based  or  laboratory-based 
programs  are  often  referred  to  as  “megascience”  programs.  These 
are  the  organized-centralized  activities  in  the  bottom  right  quadrant 
of  slide  3. 
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Each  of  these  classes  of  activities  has  different  motivations  for 
government  spending,  and  each  requires  a  different  approach  to 
management,  assessment  and  evaluation. 
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Four  Programs  Offer 
Lessons  to  be  Learned 


Superconducting  Supercollider 


Ocean  Drilling  Program 


Cooperative  Research  and  Development  Fund 
Human  Genome  Project 
Intergovernmental  Panel  on  Climate  Change 
Mars  ‘96 


RAND 


Even  if  we  agree  that  formal,  distributed  research  constitutes  an 
emerging  class  of  activities,  why  focus  on  them  specifically? 

Distributed  research  has  features  that  make  them  an  altogether  new  form 
of  collaboration.  Unlike  scientist-to-scientist  collaboration  (“bottom- 
up”  research)  or  megascience,  these  distributed  research  programs  are 
dynamic — they  use  the  global  information  infrastructure  and  they  are 
team-oriented,  task-sharing,  and  often  cross-disciplinary.  Managing 
these  programs  requires  a  different  set  of  skills  than  those  required  for 
megascience  or  laboratory-based  science. 

We  examined  four  cases  studies  in  the  course  of  this  project.  A  number 
of  candidate  projects  or  programs  initially  were  considered,  but  four 
successful  cases  were  examined  in  detail  to  identify  patterns  and  lessons 
about  organization.  As  criteria  for  creating  an  initial  list  of  cases,  we 
included  the  following  factors: 
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■  The  project  represented  one  of  a  range  of  different  motivations  for 
government  participation  in  international  cooperation  (i.e.,  foreign 
policy  goal;  global  scientific  problem;  enhance  basic  scientific 
research;  leverage  investment  in  S&T). 

■  The  program’s  research  takes  place  at  distributed  locations,  not  at  a 
central  location  or  around  a  piece  of  large  scale  equipment 

■  Government  officials  view  the  program  as  having  been  reasonably 
successful  in  reaching  its  organizational  structure  as  well  as  in 
reaching  scientific  and  policy  goals. 

■  The  program  sponsorship  includes  at  least  three  different  countries. 

We  considered  but  rejected  the  idea  of  examining  cases  of  international 
collaboration  that  were  not  entirely  successful:  For  example,  several 
people  mentioned  the  Superconducting  Supercollider  as  worthy  of  study. 
However,  unsuccessful  cases  appeared  to  have  unique  factors 
contributing  to  the  failure  of  the  effort.  In  addition,  unsuccessful  cases 
often  ended  early  in  their  tenure  and  therefore  would  not  provide  us  with 
the  information  on  operations  and  management,  which  was  a  core  part  of 
our  research  question. 

We  should  note  that  the  cases  considered  involved,  almost  exclusively, 
cooperation  among  scientifically  advanced  countries.  In  most  cases, 
scientifically  developing  countries  are  not  active  participants  at  the  level 
of  organization  and  management  of  research.  In  some  cases,  scientists 
from  scientifically  developing  countries  have  taken  part  in  research 
projects,  but  this  is  a  small  percentage  of  the  activities.  For  more 
information  on  effective  linkages  between  scientifically  advanced  and 
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developing  countries,  please  see  Guidelines  for  Research  in  Partnership 
with  Developing  Countries:  11  Principles d 

These  four  cases  were  chosen  from  a  larger  list  as  part  of  a  consultative 
process  with  our  international  partners.  Each  meets  the  criteria 
described  above.  In  addition  to  these  cases,  the  project  team  drew  upon 
its  own  experience  and  understanding  of  international  collaborations, 
and  used  some  of  our  own  experiences  to  help  develop  the  six  stages  of 
effective  linkages  detailed  in  the  following  section.  Specific  examples 
from  these  four  cases  are  included  in  the  lessons  learned. 

We  note  that  this  study  did  not  set  out  to  determine  if  these  cases  were 
successful  in  themselves,  whether  they  turned  out  good  science,  or 
whether  they  are  worth  the  investment.  These  questions  are  addressed 
in  other  studies.  This  study  looked  at  these  programs  to  pull  out  features 
of  success  so  that  policymakers  could  have  a  guide  in  hand  when 
considering  the  formation  of  similar  efforts. 

Here  is  a  brief  description  of  each  of  the  four  cases  we  examined. 

Human  Frontier  Science  Program — Foreign  policy-motivated 
collaboration.  The  Human  Frontier  Science  Program  (HFSP)  is  an 
international  research-funding  institution,  supported  by  nine 
governments,  to  support  the  neurosciences  and  molecular  biology.  The 
program  was  initiated  at  the  suggestion  of,  and  with  initial  funding  by, 
the  Japanese  government.  HFSP  provides  a  good  example  of  distributed 
research.  In  addition,  governments  had  a  strong  foreign  policy 
motivation  for  participating  in  this  program  in  an  effort  to  build 


^  Swiss  Commission  on  Research  Partnership  with  Developing  Countries  (KFPE),  Guidelines 
for  Research  in  Partnership  with  Developing  Countries:  1 1  Principles.  Bern,  Switzerland,  KFPE 
Secretariat,  1998. 


8 


goodwill  across  a  number  of  countries.  This  case  study  sheds  light  on 
how  governments  can  jointly  fund  a  central  organization  that  promotes 
funding  for  basic  research.  The  program  is  viewed  by  scientists  and 
policymakers  as  having  met  its  organization  goals  and  exceeded  its 
scientific  goals. 

Human  Genome  Project — Mission-directed,  curiosity-driven  science. 
The  Human  Genome  Project  (HGP),  an  international  consortium 
developed  to  map  and  sequence  the  human  genome,  is  an  interesting 
case  study  of  mission-directed  research.  Initiated  by  the  United  States, 
HGP  is  noteworthy  as  a  case  study  in  that  it  has  maintained  an  informal 
structure  that  has  been  adaptable,  inclusive,  and  task  oriented,  requiring 
no  diplomatic-level  international  agreements  or  formal  governing 
structure.  The  policy  goals  include  leveraging  knowledge  in  various 
laboratories  and  rapidly  (almost  instantaneously)  sharing  the  results  of 
research.  Six  governments  provide  funding  and  help  guide  this  research. 
The  program  has  exceeded  both  its  organizational  and  research  goals. 

Intelligent  Manufacturing  Systems — commercially  directed  science. 

The  Intelligent  Manufacturing  Systems  (IMS)  initiative  is  a  coordinated 
program  of  research  on  pre-competitive  manufacturing  technology. 
Suggested  initially  by  the  Japanese  government  in  1990  and 
implemented  in  1995,  six  governments  and  hundreds  of  companies  now 
take  part  in  the  research.  The  case  sheds  light  on  when  and  how 
countries  collaborate  in  sensitive  technology  arenas  where  intellectual 
property  may  be  created  and  must  be  shared.  Issues  of 
competitiveness — as  well  as  questions  of  intellectual  property  rights, 
access  to  information  and  facilities,  talent,  and  technology — make  this 
an  interesting  case  study. 

Intergovernmental  Panel  on  Climate  Change — Science  addressing 
global  problems.  The  Intergovernmental  Panel  on  Climate  Change 
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(IPCC)  is  a  coalition  of  numerous  scientific  organizations  worldwide 
organized  to  review  and  comment  on  the  scientific  research,  research 
programs,  and  environmental  policies  of  world  nations.  It  is  unlike  the 
other  cases  in  that  no  specific  research  takes  place  under  the  guise  of 
IPCC:  The  program  reviews  research  conducted  anywhere  in  the  world. 
However,  it  is  a  scientific  collaboration,  and  it  is  a  good  example  of 
nations  working  together  to  address  a  global  problem. 
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A  Number  of  Policy  Questions  Arise  over  the 
Course  of  a  Program 

H  Sdence&  Technoloav  Policy  Institute _  _ 


been  \mthtfB  effort? 

Is  this  prog'am  reaching  goals  we  set  out? 

V\ihat  are  the  key  issues  we  need 

:o  negotiate,  nxinita? 

l-b«  shoUd  this  prog'arri  be  organized  and  rrariaged? 

\Adch  parties  shoJd  find  this,  nationally,  intemalionally? 

Wn  are  ttiie  key  actors  to  include? 

Should  we  initi 

jte'partidpate? 

Life-span  of  ai  International  S&T  Cdlaborstive  / 

^^o^am 
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Across  the  life  span  of  these  programs,  policymakers  face  a  number  of 
questions  (some  of  these  important  questions  are  illustrated  in  the  figure 
above  the  life-span  line).  However,  the  bars  partly  outside  the  vertical 
lines  illustrate  those  decisions  that  are  largely  political.  It  is  difficult  to 
decide  how  to  derive  generalizable  lessons  learned  at  the  political  level. 
Nevertheless,  it  is  possible  to  say  why  existing  programs  were 
successfully  operated,  so  this  study  focuses  on  lessons  learned  from  the 
organization  and  management  of  successful  collaborations. 

The  first  step  in  the  process  involved  collecting  published  information 
about  the  case  studies.  We  identified  policy  statements,  program 
announcements,  legislation,  and  other  program  materials  describing  the 
role  of  governments  in  the  international  cooperative  program  in  the  four 
case  studies.  The  second  step  involved  identifying  and  surveying  key 
administrative  personnel  and  policymakers  who  took  part  in  the 
initiation  or  administration  of  the  program. 
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In  interviews,  we  sought  to  gather  information  about  (1)  the  motivation 
and  intent  of  the  government’s  participation  in  the  program,  (2)  the 
organization  and  structure  of  the  program,  (3)  how  knowledge  was 
created  and  managed,  and  (4)  whether  and  what  evaluation  tools  and 
methods  were  used. 

We  interviewed  more  than  30  scientists  and  policymakers,  some  face-to- 
face  and  others  by  phone.  The  questions  we  asked  included: 

■  What  was  the  original  intent  of  the  government’s  involvement  in  this 
collaboration?  Was  the  original  intent  carried  forth  into  actual 
planning  and  execution?  If  not,  why  did  the  original  intent  fail  to 
instruct  the  organization  of  the  project? 

■  Within  the  original  intent,  what  motivating  factor  was  more 
important  to  the  government’s  goal:  curiosity-driven  science, 
foreign  policy  concerns,  industrial  competitiveness,  a  specific 
mission  of  the  government,  or  another  factor? 

■  Was  the  organization  of  the  government’s  administration  of  this 
program  arranged  to  suit  national  government  requirements  or  to 
enhance  international  linkages? 

■  What  was  successful  about  the  government’s  organization  and 
administration  of  this  program?  Why? 

■  What  features  of  the  government’s  administration  and  organization 
did  not  work  well?  Why? 

■  Did  the  program  succeed  in  attracting  excellent  scientific  proposals? 


12 


■  How  should  collaborations  be  evaluated,  especially  given  the 
difficulty  of  quantifying  basic  research  outcomes? 

The  responses  to  these  questions  form  the  basis  of  the  analysis 
developed  in  this  briefing. 
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Six  Key  Stages  Emerged 

H  Sdence&  Technoloav  Policy  Institute _ 
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II.  SIX  KEY  STAG  ES  EM  ERG  ED 


A  pattern  emerged  from  the  case  studies.  Each  of  the  programs 
presented  organizers  with  the  questions  we  introduced  above.  Once  a 
scientific  need  was  established  as  worthy  of  consideration,  we  found  that 
the  initiation,  administration,  and  management  of  the  four  case  studies 
could  be  characterized  in  six  stages: 

an  initial  decision  and  design  stage,  determining  whether  it  makes  sense 
to  create  a  new  program  on  scientific  grounds  and  whether  key  missions 
of  government  will  be  served 

an  effort  to  identify  and  contact  key  actors  from  different  interest 
groups  (“stakeholders”)  in  the  political,  scientific,  and  international 
communities 


> 


RAND 
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a  delicate  stage  of  identifying  initial  and  then  sustainable  funds  and  the 
mechanisms  to  share  resources 


an  effort  to  determine  the  organization  and  management  structure  of  the 
actual  research  (or  research  review)  process 

the  need  to  identify,  negotiate,  and  adjudicate  key  issues  such  as  sharing 
of  intellectual  property  or  protecting  the  rights  of  participants 

identification  of  the  methods  to  evaluate  progress  and  communicate  the 
benefits  of  the  activities  to  a  range  of  interested  stakeholders. 

These  stages  are  described  in  more  detail  in  the  following  slides. 
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Political  and  Scientific  Motivations  Both  Play  a  Role 

—  SdenceSi  Technoloav  Policy  Institute _ _ 

Governments  initiate  or  join 
coiiaborations  to: 

-  create  knowledge,  general  or  specific 

-  share  costs  of  expensive  research 

-  access  resources 

-  gain  goodwill 

-  support  specific  mission 

Scientists  join  coiiaborations  to: 

Should  we  initiate/participate?  -  access  scarce  funding 

-  gain  access  to  foreign  labs 

-  link  directly  with  foreign  partners 

-  share  information  in  real-time 

-  enhance  creativity  of  research 

*  RAND 


From  this  and  other  studies  of  international  collaboration  we  have 
conducted,  we  find  that  governments  and  scientists  are  both  interested  in 
collaboration  for  interlocking,  but  often  for  different  reasons. 
Governments  initiate  or  participate  in  collaboration  most  often  to  meet 
policy  goals.  They  seek  to  create  knowledge,  of  course,  but  in  a  national 
scientific  system  as  large  as  the  United  States,  knowledge  creation  does 
not  usually  require  transnational  linkages.  Using  international 
collaboration  to  reach  policy  and  scientific  goals  must  carry  additional 
benefits  for  governments,  ones  that  include  sharing  the  costs  of  research, 
as  in  the  case  of  IMS’s  research  into  software  that  controls  advanced 
manufacturing  equipment;  gaining  access  to  resources  that  might  not 
otherwise  be  available  such  as  genome  data;  supporting  a  specific 
mission  such  as  environmental  protection;  and  generally  creating 
goodwill  through  science. 

Scientists  also  welcome  collaborations  when  it  helps  them  to  effectively 
answer  a  specific  research  question.  Even  though  international 
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collaborations  have  higher  transaction  costs  than  local  collaboration  or 
simply  working  alone,  scientists  see  a  benefit  to  international 
collaboration  when  it  funds  research  that  might  not  otherwise  be  funded, 
such  as  some  of  the  brain  research  funded  by  HFSP.  Formal  programs 
help  open  the  doors  to  foreign  laboratories  that  otherwise  may  have  been 
difficult  to  access,  and,  at  times,  collaborative  programs  can  help 
scientists  find  partners  with  similar  research  interests.  Sharing 
information  in  real  time  is  often  emphasized  by  the  formal  programs, 
giving  scientists  a  way  to  keep  on  top  of  data — this  has  been  the  case  in 
the  data-sharing  polices  of  HGP.  Scientists  also  report  that  working 
with  foreign-trained  researchers  gives  them  new  insights  into  how  to 
think  about  science.  Science  is  about  creativity;  these  linkages  enhance 
creative  thinking. 

During  the  initial  phase,  these  programs  had  a  number  of  features  that 
helped  them  get  off  the  ground.  In  all  the  cases,  the  organizers  worked 
to  clearly  articulate  a  scientific  and  political  benefit  to  U.S.  government 
participation  in  these  programs.  This  may  not  have  happened  initially, 
but  as  champions  and  organizers  interacted  with  other  decisionmakers 
and  stakeholders,  they  developed  and  improved  materials  to  demonstrate 
the  usefulness  of  the  activity.  In  the  best  cases,  they  persuaded  key 
groups  to  become  involved,  and  early  coordination  occurred  across 
relevant  U.S.  government  agencies.  (In  cases  where  coordination  did 
not  take  place,  many  reported  that  this  was,  in  hindsight,  a  mistake — 
coordination  should  have  happened  in  the  initial  decision  and  design 
phase.)  Policymakers  also  cited  the  need  for  a  clear  rationale  for 
government  sponsorship  and  national  participation  in  the  collaboration 
as  an  important  feature  of  any  decisionmaking  process. 
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A  Written  Agreement  is  Often  Needed 


B  Science  &  Technoloav  Policy  I  nslitute 


Agreements  can  be  negotiated  and  signed 
at  a  number  of  ieveis: 

treaty-level  agreement 

memorandum  of  understanding  or  agreement 
exchange  of  letters  between  agencies 

project  agreement  between  directorate  or 
bureau-level  officials 

equipment-sharing  agreements  to  facilitate 
cross-border  exchange 


The  cases  we  examined  did  not  start  ojfwith  high-level  agreements; 
agreements  were  made  at  the  program  level 
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When  governments  get  involved  in  organizing  and  managing 
international  programs,  formal  agreements  often  establish  the 
groundwork  for  interaction.  There  is  no  single  template  for  the  level  at 
which  an  agreement  is  negotiated  and  none  for  its  formality.  Notably, 
the  four  cases  we  examined  did  not  require  or  generate  the  need  for  a 
high-level  govemment-to-govemment  agreement,  such  as  a  treaty-level 
agreement,  a  bilateral  S&T  agreement,  or  even  a  less  formal 
memorandum  of  understanding. 

HFSP  and  IMS  participants  used  an  informal  exchange  of  letters  upon 
agreeing  to  cooperate.  IPCC  was  conducted  under  existing  United 
Nations  treaties.  While  no  formal  agreements  were  found  necessary  to 
manage  HOP,  there  was  an  agreement  on  data  exchange  where 
international  partners  agreed  to  formalize  the  conditions  of  data  access, 
including  release  of  sequence  data.  Participating  representatives  made 
agreements  about  program  operations  as  well  as  guidelines  for  sharing 
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funds  and  intellectual  property,  but  these  were  largely  working 
documents  that  emerged  from  boards  of  directors  or  scientific 
committees.  These  served  as  the  “agreements”  guiding  participation  and 
research. 
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Key  Questions  include  Goal  Setting 


B  Science  &  Technoloav  Policy  Institute 


Stage  One:  Initial  Decision  and  Design 

Does  this  collaboration  meet  national  goals? 

Will  this  collaboration  promote  excellent  science? 

Will  this  activity  support  other  foreign  policy  goals? 

Do  we  have  a  clear  problem  statement? 

Is  the  rationale  for  this  activity  written  down? 

Is  there  a  problem  that  can  be  addressed  and  that  can  be  solved? 
Are  key  agencies  aware  of  and  supportive  of  this  activity? 

Are  key  scientific  groups  supportive  of  this  activity? 

What  is  the  value  added  of  government  participation? 
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The  initial  decision  and  design  phase  brings  us  back  to  the  first  question 
on  the  list  of  issues  facing  policymakers:  Should  we  initiate  in  an 
international  collaboration  or  participate  in  one  being  formed?  For  the 
United  States,  this  is  a  crucial  question.  The  U.S.  R&D  system  is  huge, 
accounting  for  $230  billion  annually,  according  to  the  National  Science 
Board.  Because  of  the  sheer  size  of  the  scientific  community,  it  is 
relatively  easy  to  find  a  collaborator  within  the  United  States.  So,  for 
the  U.S.  government  to  consider  sponsoring  a  collaborative  project  or 
program,  there  should  be  a  compelling  reason  to  get  involved 
internationally  rather  than  fund  R&D  within  the  United  States.  In  the 
opinion  of  one  policymaker,  “You  have  to  ask:  Does  the  collaboration 
allow  you  to  do  things  you  would  not  do  otherwise?  If  so,  then  the  U.S. 
should  definitely  take  part.”  We  found  a  number  of  reasons  why  the 
United  States  participated  in  the  programs  we  studied,  as  indicated 
above.  The  motivation  for  participating,  however,  was  often  complex — 
the  project  needed  to  have  scientific  as  well  as  political  and,  often. 
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economic  benefits.  Usually,  a  specific  case  had  a  number  of  interrelated 
and  overlapping  motives  that  spurred  U.S.  government  interest  and 
willingness  to  participate. 
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Contacts  are  Needed  in  Policy,  Science, 
and  International  Circles 

^  ^  A  stage  2:  A 

(  )  \  Key  People  J 

Three  groups  are  brought  into  the 
initial  stages  of  decisionmaking: 

-  policymakers  in  scientific  and  mission 
agencies 

-  respected  scientists  and  engineers, 
nationaiiy  and  internationally 

-  political  contacts  in  foreign  governments  are 
tapped 

Who  are  the  key  actors  to 
include? 

...Ignoring  any  one  of  these  groups  causes 
problems... 

11 

RAND 

Each  of  the  case  studies  we  examined  had  “champions” — individuals 
with  credibility  who  promoted  U.S.  participation  in  the  activity.  In 
addition  to  helping  get  a  project  off  the  ground  by  convincing  the 
scientific  and  policy  communities  that  it  was  a  worthwhile  activity, 
champions  also  continued  to  take  an  interest  in  the  program  over  time. 
As  one  policymaker  told  us:  “You  need  committed  senior  people  who 
will  go  to  meetings  and  will  stay  with  it.  Trust-building  is  a  key  part  of 
this  activity.”  Champions  promoted  the  project  to  such  key  stakeholder 
groups  as  the  U.S.  Congress  and  scientific  societies.  The  champions 
were  identified  in  one  of  two  ways:  (1)  they  were  part  of  the  initial 
scientific  or  policy  team  proposing  the  joint  project,  or  (2)  they  were 
asked  by  one  of  the  early  promoters  to  associated  their  name  with  the 
project. 
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Key  Questions  include  Whether 
a  “Champion”  is  Involved 

B  Science  &  Techndoav  Pdicv  I  nslitute _ 


Stage  Two;  Identifying  Key  Actors  and  Participants 
Does  this  activity  have  a  champion? 

Are  key  government  people  on-board  to  aid  decision-making? 

Are  there  multiple  benefits  -  political,  scientific,  commercial? 

Can  we  effectively  communicate  to  Congress  the  value  of  this 
activity? 

Are  respected  scientists  part  of  the  planning  team? 

Are  individuals  knowledgeable  about  legal  and  institutional  issues 
part  of  the  planning  team? 

Which  countries  should  be  invited  to  participate? 

Do  other  participating  countries  also  have  champions? 

RAND 


Connecting  with  all  the  stakeholders  and  ensuring  that  each  group  had 
some  input  were  features  that  were  identified  across  all  the  case  studies 
as  important.  In  several  cases,  a  key  group  was  ignored — an  oversight 
that  led  to  problems.  We  were  told  that  the  involvement  of  any  and  all 
interested  groups  is  important. 

Decisions  about  appropriate  and  interested  collaborators  take  place,  at 
the  very  least,  at  three  different  levels  during  the  design  phase:  (1) 
within  the  federal  government,  (2)  within  the  international  S&T 
community,  and  (3)  within  the  group  of  scientists  and  program  managers 
who  establish  the  initial  organization. 

At  the  federal  government  level,  questions  about  appropriate  actors 
include: 

■  Who  within  the  federal  government  should  be  involved  in  initial 
design  and  longer-term  oversight? 
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■  Which  (nationally  based)  scientists  or  engineers  should  be  invited  to 
discussions  about  U.S.  interests? 

■  At  international  meetings,  decisions  were  raised  to  a  different  level 
and  included  questions  such  as: 

■  Which  countries  should  be  involved  in  the  design  and  management 
of  these  programs? 

■  How  do  we  set  criteria  for  allowing  new  sponsors  or  other  new 
members  to  participate? 

At  the  initial  program  design  meetings,  questions  about  actors  focused 

on  the  following: 

■  Should  private- sector  researchers  be  involved? 

■  Should  the  focus  be  on  established  researchers  or  on  younger 
researchers? 

■  What  types  of  scientists  should  be  encouraged  to  apply  for  support 
or  to  engage  in  collaborative  research? 

■  Should  collaboration  take  place  at  the  laboratory  level  (i.e.,  support 
goes  to  a  lab  and  is  distributed  at  the  site)  or  at  the  scientific  level 
(i.e.,  scientists  receive  direct  support  and  can  work  at  the  laboratory 
of  their  choice)? 
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Ensuring  Funding  Commitments  is  a  Delicate  Step 

^  ^mce&  Techndocn/  Policv  1  nsiitute 

Funding  sources: 

stages  - 

\Identify  Funding^  J 

solely  government-funded 
government-industry  jointly  funded 
Industry-only  funding 

International  organizations 

combined  sources  of  funding  (Including 
foundations,  universities) 

Funding  mechanisms: 

Which  parties  shouid  fund  this 

“single-pot”  combined  funding 

program,  nationaiiy, 
internationaiiy? 

dispersed,  participant-funded 

13 

RAND 

Funding:  Perhaps  no  issue  is  quite  as  divisive  and  troublesome  as 
identifying  who  will  fund  a  new  program,  how  and  whether  the  funds 
will  be  shared  and  allocated,  how  to  sustain  funding,  and  how  to  ensure 
that  benefits  accrue  to  those  providing  the  support.  Funding  issues 
occur  at  every  level.  First,  they  occur  at  the  level  of  the  government 
initiating  the  program:  Which  agencies  will  fund  this  effort,  or  will  the 
legislature  be  asked  to  appropriate  funds?  How  can  any  group  be 
convinced  that  relinquishing  control  over  national  funds  to  an 
international  body  is  a  good  way  to  spend  its  money?  Once  these 
questions  are  answered  (often  helped  by  the  program  champions), 
deciding  whether  the  funds  will  be  co-mingled  with  those  of  other 
governments,  or  whether  funding  will  be  coordinated  with  others, 
becomes  an  issue.  Should  private  or  philanthropic  funds  be  included? 
All  these  issues  came  up,  and,  just  as  we  found  with  other  questions, 
there  is  no  “right  answer.”  There  are  only  methods  for  funding,  and  lots 
of  questions  that  should  be  considered. 
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Key  Questions  include  Sustainability 

B  Science  &  Techndoav  Pdicv  I  nslitute _ 


Stage  Three;  Funding  Sources  and  Mechanisms 

Do  we  have  sustainable  funding  for  the  collaboration? 

Are  all  partners  contributing  in  accordance  with  agreements  on 
distribution  of  costs  and  benefits? 

Will  we  only  accept  government  funding? 

Will  philanthropic  funds  be  sought? 

Can  industry  contribute  funding?  Under  what  conditions? 
Should  all  funds  be  combined  into  a  single  pot? 

Will  each  country  pay  its  own  way? 

Will  the  collaboration  provide  grants  to  scientists? 

Will  peer  review  determine  the  allocation  of  funds? 


RAND 


Maintaining  a  stable  source  of  funding  is  clearly  a  baseline  requirement 
for  any  international  program.  Maintaining  a  clear  line  of  funds  has 
been  one  of  the  central  challenges  facing  these  cases.  Funding  can  be 
contributed  by  a  single  source  (i.e.,  governments  only),  or  can  be  a 
hybrid  of  different  sources.  The  cases  we  examined  included  all  types 
of  funding  sources. 

The  second  funding  issue  reported  to  us  is  ensuring  that  all  partners  are 
bringing  equitable  contributions  to  the  effort  as  well  as  obtaining 
benefits  in  line  with  their  contribution  (even  if  these  are  intangible 
benefits).  As  one  observer  told  us,  “In  order  to  have  a  collaboration  that 
is  ‘win-win,’  all  partners  must  derive  a  benefit  that  is  commensurate 
with  their  contributions.”  Should  the  host  country  or  primary  funding 
country  have  special  voting  rights  or  other  privileges? 

Alternatively,  there  may  be  cases  where  a  partner  or  applicant,  such  as  a 
scientist  from  a  developing  country,  cannot  bring  an  equal  financial 
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contribution.  This  may  be  acceptable  to  participants.  However, 
discussing  these  issues  of  equitable  funding,  sharing,  or  subsidy  is 
important  during  the  early  stages  of  planning  and  design,  an  issue  cited 
by  several  people  as  important  for  smooth  operations  downstream. 

What  should  the  group  do  if  one  country  does  not  pay  their  share  or 
decides  to  withdraw  from  the  project? 

Within  specific  projects,  a  decision  should  be  made  whether  funding 
will  be  used  to  initiate  activities  or  leverage  existing  ones.  As  one 
champion  told  us,  “If  there  is  not  enough  money  dedicated  to  new 
projects,  then  scientists  just  collaborate  around  work  they  already  have 
under  way.  If  this  is  going  to  be  new  and  value-added,  you  have  to  fund 
projects  adequately.” 

Nevertheless,  in  two  cases  we  examined,  the  program  and  goal  were  to 
leverage  and  share  existing  research.  Funding  mechanisms  include 
grants  and  fellowships,  contributions  to  the  operation  of  laboratories  or 
the  building  of  databases,  funding  for  industrial  research,  and  travel 
funds  to  attend  conferences  and  meetings. 

Within  national  governments,  the  question  of  who  will  pay  can  become 
important  and  highly  political.  U.S.  government  agencies  rarely  have 
budget  line  items  that  allow  them  to  commit  funds  directly  to 
collaboration.  As  a  result,  funds  are  allocated  from  specific  mission- 
oriented  programs.  Some  program  managers  in  U.S.  government 
agencies  are  reluctant  to  commit  funds  to  international  activities  because 
they  may  lose  some  control  over  the  activity,  or  it  may  be  viewed  as 
“foreign  aid”  in  another  form.  Moreover,  U.S.  government  programs  are 
funded  on  a  year-to-year  basis,  so  the  question  of  whether  funds  will  be 
available  to  fund  a  multiyear  program  is  not  always  known  ahead  of 
time. 
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Sometimes  a  decision  about  how  a  program  will  be  funded  depends  on 
the  structure  of  agency  budgets.  It  is  far  easier  if  agencies  can  pay  for 
activities  out  of  budgeted  programs  rather  than  needing  to  transfer  funds 
to  a  third  party.  Even  so,  at  least  one  case  we  examined,  HFSP, 
government  funds  are  contributed  to  a  “single  pot”  of  funding  that  is 
distributed  at  the  international  program  level.  More  often,  funds  for 
collaboration  are  dispersed  by  the  agencies,  and  each  participating 
country  or  partner  funds  its  own  participation  in  the  collaboration. 
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Publicizing  Mission,  Inviting  Participants 
are  Key  Administrative  Steps 

H  Sdence&  Techno/oav  Policy  Institute _  _ 


Operating  procedures  include: 

establishing  scientific  missions  and  goals 
determining  membership  and  voting  rules 

-  deciding  on  a  research  agenda 
establishing  endorsement  criteria 
determining  eligibility  for  funding  or  inclusion 

-  inviting  proposals  or  comments 

-  assessing  quality  of  requests  for  funding 

-  aiding  junior  scientists 

-  monitoring  on-going  projects,  processes 


> 


RAND 


Once  decisions  are  made  about  actors  and  champions  at  the  different 
levels  of  decisionmaking,  the  next  step  is  to  determine  how  the  new 
collaboration  will  be  organized,  as  well  as  how  and  where  it  will  be 
staffed,  managed,  and  evaluated.  In  several  of  the  cases,  these  decisions 
were  made  at  the  program  organization  level,  and  U.S.  government 
officials  had  little  input  into  the  staffing  and  management  decisions.  In 
two  of  our  cases,  however,  U.S.  government  officials  worked  closely 
with  organizers  to  make  decisions  about  organization  and  management 
structure. 

In  each  of  the  cases,  some  combination  of  the  factors  listed  in  the  figure 
contributed  to  success.  These  included  a  flexible  structure  that  could 
adapt  to  new  information  about  the  impact  the  program  was  having.  As 
one  told  us,  “The  simplicity  of  approach  has  been  a  large  part  of  the 
success.”  As  scientific  research  developed,  the  mission,  scope,  or 
participants  often  changed  or  included  new  areas  of  research  or  new 
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targets.  In  order  to  remain  current  and  relevant,  program  managers 
solicited  input  from  a  number  of  sectors.  They  were  able  to  use  new 
information  to  help  keep  the  programs  focused  on  important  and  useful 
questions.  Each  of  these  cases,  for  different  reasons,  avoided  being 
heavily  influenced  by  political  entities.  The  focus  remained  on  the 
scientific  or  technical  question  and  scientists  or  engineers  offered 
guidance  to  keep  the  program  moving  in  the  right  direction. 

Establishing  a  mission  statement  for  the  program,  one  delineating 
specific  goals,  was  important  to  the  life  of  the  new  organization.  We 
found  that,  while  establishing  scientific  missions  and  goals  was  an 
important  component  of  the  early  design  phase,  S&T  produced  new 
knowledge  that  constantly  challenged  the  programs  to  revise  their  vision 
and  their  research  agenda.  Each  of  the  programs  adapted  its  scientific 
missions  to  new  information.  Eikewise,  determining  eligible  members 
of  boards,  determining  the  technical  people  who  should  be  included  in 
program  activities,  and  determining  who  should  receive  funds  or  other 
support  were  issues  that  continually  arose  as  these  programs  undertook 
day-to-day  operations.  These  questions  were  dealt  with  at  all  levels  of 
organization:  from  boards  of  directors  down  to  the  scientists  or 
engineers  and  staff  managing  daily  interactions.  The  same  is  true  for 
establishing  endorsement  criteria  and  determining  eligibility  for  funding 
or  participation.  These  issues  were  not  always  worked  out 
harmoniously.  In  fact,  in  many  cases  there  were  considerable 
disagreements  concerning  governance  and  operating  issues. 
Nevertheless,  the  programs  had  the  governing  structure  and 
communications  linkages  in  place  to  provide  a  forum  for  adjudicating 
these  issues. 


30 


Scientists  Play  a  Key  Role  in  Governance 

H  Sdence&  Technoloav  Policy  Institute _ 


Program  governance  methods  vary, 

but  can  include: 

boards  of  directors  or  trustees 

boards  or  councils  of  scientists  and  other 
peer  reviewers 

consortia  of  participants 

international  and  non-governmental 
organizations 

steering  committees 
observer,  stakeholder  bodies 


RAND 


Each  of  the  cases  we  examined  had  different  governing  mechanisms, 
and  each  appeared  to  work  well  for  that  case.  For  example,  HFSP  has  a 
board  of  trustees  with  a  mix  of  scientists  and  policymakers,  as  well  as  a 
council  of  scientists.  A  small  consortium  of  participants  representing 
the  institutions  from  each  of  six  countries  governs  HGP.  IMS  is 
governed  by  an  international  steering  committee  composed  of  delegates 
from  industry  and  research  institutions  of  each  participating  region. 
IPCC  has  a  very  simple  structure  but  a  highly  elaborate,  bottom-up 
review  process  to  produce  its  assessment  reports.  The  IPCC  secretariat, 
bureau,  and  working  groups  are  responsible  mainly  for  organizational 
and  coordination  matters  to  facilitate  the  review  process.  The  nature  of 
the  governing  structure  depends  on  the  type  of  inquiry  being  pursued 
and  what  kind  of  organizational  structure  is  needed  to  facilitate  the  S&T 
collaborations. 


31 


Administration  Fits  the  Needs, 
Goals  of  the  Program 

H  SdenoeSc  TechnoloqvPolicv Institute _ 


Program  administration  can  take  one 
of  a  number  of  forms,  inciuding: 

a  permanent  secretariat  or  executive 
office  (HFSP) 

a  rotating  secretariat,  moving  among 
partner  nations  (IMS) 


regionai,  distributed  administration, 
iooseiy  coordinated  (HGP,  IMS) 

How  should  this  program  be  -  intergovernmental  or  scientific  panel 

organized  and  managed?  (IPCC,  HFSP) 


RAND 


Each  of  the  cases  we  studied  had  a  different  way  of  operating  the 
program.  HFSP  has  a  permanent  secretariat  who  acts  as  an  executive 
office.  HGP  has  a  loosely  coordinated  structure,  where  the  U.S.  National 
Institutes  of  Health  generally  manages  the  collaboration  working  closely 
with  the  Wellcome  Trust  and  other  key  institutions  involved  in  the 
project.  IMS  has  a  rotating  interregional  secretariat  as  well  as  regional 
secretariats  who  administer  the  program  and  projects.  IPCC  has  a 
secretariat  to  coordinate  organizational  matters;  three  working  groups 
under  a  bureau  headed  by  the  IPCC  chair  coordinate  scientific 
assessment.  The  nature  of  the  scientific  or  technical  inquiry,  as  well  as 
the  interests  of  the  participants,  helped  to  determine  how  the  program  is 
administered. 

Factors  that  influence  organizational  features  are:  (1)  the  nature  of 
funding  allocation  and  (2)  the  method  of  data-sharing.  When  a  central 
pot  of  funds  is  involved,  a  centralized  secretariat  may  be  needed  as  a 
place  where  potential  grantees  can  contact  for  information,  as  is  the  case 
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with  HFSP.  Conversely,  when  each  participating  country  is  funding  its 
own  research,  data-sharing  may  be  the  most  important  function.  This 
does  not  require  a  central  office;  however,  it  requires  a  single  collection 
and  dissemination  point  for  data.  The  case  of  HOP  shows  how  a 
collaborative  program  can  share  data  effectively  over  the  Internet. 
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Key  Questions  include 

Location  of  Coordinators 

- > 

Stage  Four:  Organization  and  Management  Stmcture 

^  stage  4 

- ^  \  Determine 

^..^organization 

)  Does  the  collaboration  require  a  central  coordinating  office?  If  so, 
where  should  it  be  located? 

Which  countries  will  have  voting  rights/controlling  authority? 

Does  the  collaboration  we  are  planning  have  a  flexible  and  adaptable 
decision-making  stmcture? 

Are  we  allowing  the  needs  and  demands  of  science  to  determine  the 
organization  of  the  collaboration? 

Does  the  stmcture  facilitate  input,  feedback,  and  working 
relationships  among  participants? 

Have  private  and  non-governmental  groups  been  invited  to 
participate  or  share  views? 

18 
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How  the  program  will  be  organized  and  coordinated  is  a  key  issue  that 
arose  in  each  of  the  cases.  As  noted  above,  not  all  collaborations  require 
a  central  office,  but  a  coordinator  is  needed  for  each  program. 
Determining  which  countries,  companies,  or  groups  are  “in” — that  is, 
allowed  to  become  full  members — and  whether  new  parties  will  be 
allowed  to  join  is  a  particular  problem  for  successful  activities.  Once 
the  program  begins  showing  results,  other  parties  will  naturally  want  to 
join.  How  will  this  be  decided?  Each  of  the  cases  we  looked  at  decided 
this  differently,  some  harkening  back  to  their  mission  statement  as  a  way 
to  retain  the  original  order,  others  widening  the  circle  to  allow  new 
members.  Programs  with  limited  funds  are  less  likely  to  accept  new 
members  unless  new  funds  come  along  as  well.  Programs  that  share 
proprietary  data  are  also  less  likely  to  welcome  new  members  who  have 
not  yet  signed  on  to  the  protocols  of  information  exchange  and 
protection. 
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Negotiating  Issues  Takes  Place 
Throughout  the  Life  of  the  Program 

H  SdenaeA  TechnoloavPolicv Institute _ _ 


Early  in  the  program,  identify, 
negotiate  major  issues: 

intellectual  property  rights  agreements 

establishing  appropriate  legal 
arrangements,  if  necessary 

addressing  access  and  information 
sharing  issues 


What  are  the  key  issues  we  need 
to  negotiate,  monitor? 


publication  rights 
access  to  facilities 
communication  linkages 


RAND 


In  each  of  the  cases  we  examined,  organizers  and  participants  faced  key 
issues  that  are  common.  Most  reported  that  identifying  and  dealing  with 
key  issues,  particularly  early  in  the  process,  can  be  important  for 
successful  operation.  Questions  about  protecting  existing  and  newly 
created  intellectual  property  were  key  to  a  number  of  the  cases  that  we 
examined.  Whether  or  not  the  program  wanted  credit  when  results  were 
published,  how  to  manage  access  to  facilities,  and  how  to  share  data 
were  also  issues  that  arose  as  the  programs  began  operation. 

No  matter  what  the  specific  issue,  all  of  the  program  managers  reported 
that  issues  arise  throughout  the  life  of  the  program.  To  be  ready  for 
these  issues,  a  clear  and  effective  method  of  communicating — top-down 
and  across  the  different  activities  being  sponsored — was  a  key  factor 
affecting  success.  Cultural  factors  certainly  affect  operations  and  should 
be  illuminated,  acknowledged,  respected,  and  understood.  Face-to-face 
meetings  are  required  at  frequent  intervals  for  organizers  as  well  as  for 
researchers.  As  one  program  manager  told  us,  “The  collaborators  in  our 
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program  meet  face-to-face  once  a  year.  This  is  part  of  their  budget.” 
New  information  technologies  enhance  communication,  but  cannot  be 
relied  on  as  the  sole  means  of  communication:  A  diverse  set  of 
communication  methods  is  needed  to  ensure  success. 
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Key  Questions  include 

Sharing  Intellectual  Property 

O'-'O 

Stage  Five:  Determination  of  Key  Issues  p 

A  stage  5  A  ^ 

Flow  will  we  handle  prior  intellectual  property? 

VNegotiate  key  issues J  • 

Have  rules  been  negotiated  concerning  intellectual  property  rights  ■ 
and  sharing  of  data?  1 

Can  we  arrange  periodic  meetings  to  share  scientific  findings?  1 

Will  pubhcations  resulting  from  this  work  acknowledge  support? 

Can  we  easily  access  each  others’  research  facilities? 

Can  scientific  equipment  be  shipped  across  borders? 

How  do  we  acknowledge  important  cultural  differences? 

20 

RAND 

A  sample  of  the  key  issues  and  questions  that  were  reported  to  us  are 
included  here.  The  figure  highlights  the  role  of  sharing  of  intellectual 
property.  In  any  international  venture,  the  question  of  sharing  and 
protecting  the  rights  of  originators  to  use  the  knowledge  they  create  will 
be  an  issue.  Questions  about  publication  rights  should  be  handled  early 
in  any  coordinated  activity.  Acknowledging  cultural  differences  and 
their  effect  on  the  functioning  and  outcomes  of  a  program  was  a  feature 
that  several  people  mentioned  was  important.  Partners  should  be  seen 
and  experience  themselves  as  equal  partners.  There  should  be  a  common 
agreement  among  participants  on  the  scientific  and  policy  goals  that  are 
motivating  the  organization. 
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Evaluation  is  Important  at  the  Political 
and  the  Scientific  Level 

B  Science  &  Techndoav  Pdicv  I  nslitute _ 

Features  that  improve  programs: 

benefits  accrue  equally  to  participants 
benefits  are  effectively  articulated 

evaluation  criteria,  ideally  built  into  the 
project  and  monitored 

self-evaluation  by  project  participants 
assists  with  feedback 

an  “exit  strategy”  or  end-point  for  the 
collaboration  is  clearly  defined 

Is  this  program  reaching  goals 
set  out  for  it? 


RAND 


Organizers  told  us  that  being  able  to  clearly  state  the  outputs  and 
outcomes  of  research  is  a  key  feature  of  success.  Nevertheless,  this  is  a 
feature  that  did  not  receive  adequate  attention  at  the  design  phase  in 
several  of  the  cases  we  examined,  according  to  policymakers 
interviewed.  A  number  of  policymakers  and  organizers  recommended 
building  measures  of  outputs  and  outcomes  into  the  program  design  and 
requiring  responses  from  scientists  as  to  how  collaboration  helped  them 
reach  goals. 

In  a  larger  set  of  questions,  a  concern  expressed  by  some  federal 
government  agency  officials  was  whether  a  clear  “exit  strategy”  had 
been  devised  for  the  U.S.  government.  For  example,  at  what  point  has 
the  U.S.  government  reached  its  goals  and  therefore  bows  out  of 
collaborating?  This  feature  was  lacking  in  some  of  the  cases  we 
examined,  but  it  was  cited  as  a  factor  that  should  be  carefully  considered 
in  framing  a  new  collaboration. 
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In  two  cases  we  examined,  an  independent  evaluation  of  the 
effectiveness  of  the  program  was  conducted.  This  created  opportunities 
in  each  case  to  make  adjustments  to  the  program  based  on  feedback.  A 
formal,  external  evaluation  helped  champions,  policymakers,  and 
program  managers  make  their  case  to  other  stakeholders,  such  as 
legislative  bodies,  for  changes  or  additional  funding  as  appropriate. 
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Key  Questions  include  Building  Feedback  Loops 

B  Science  St  Techndoav  Pdicv  I  nslitute _ 


Stage  Six;  Benefits  Assessment  and  Evaluation 


How  can  benefits  be  effectively  articulated  and  to  whom  will 
they  he  provided? 

Can  evaluation  criteria  he  built-in  to  monitor  individual  projects 
or  the  entire  program? 

How  often  will  assessments  he  carried  out? 

What  are  the  feedback  mechanisms  within  the  program? 

Can  project  participants  assist  by  providing  feedback? 

Is  there  provision  for  periodic  evaluation  leading  to  renewal  or 
termination? 


RAND 


Creating  the  mechanisms  by  which  the  outputs  and  outcomes  of  research 
will  be  tracked  and  monitored  requires  building  feedback  loops  within 
the  program,  according  to  several  people  we  spoke  to.  “It  helps  to  know 
how  the  projects  are  doing  in  progress,  not  just  at  the  end,”  one  program 
manager  said.  In  order  to  do  this,  periodic  meetings  help  to  organize  the 
thoughts  of  researchers  and  provide  a  forum  to  discuss  research  progress 
as  well  as  how  well  the  program  is  facilitating  the  cooperative  activity. 

Criteria  for  evaluation  can  be  built  into  the  program  in  a  way  that  helps 
to  monitor  administration  of  the  program  and  research  activities. 
Ongoing  monitoring  helps  make  midcourse  corrections  in  cases  where 
specific  policies  are  not  working  well,  or  where  a  project  is  not  going 
well.  Signposts  of  progress  can  include,  at  the  administration  level, 
whether  grants  are  being  processed  on  time,  whether  new  applications 
are  increasing,  whether  participants  are  providing  feedback  in  a  timely 
fashion,  etc.  The  specific  signposts  of  progress  will  differ  with  each 
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program,  but  the  point  is  to  identify  and  build  these  in  at  the  beginning 
of  a  new  activity. 
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Key  Questions  include  Building  Feedback  Loops 

B  Science  St  Techndoav  Pdicv  I  nslitute _ 


Stage  Six;  Benefits  Assessment  and  Evaluation 


How  can  benefits  be  effectively  articulated  and  to  whom  will 
they  he  provided? 

Can  evaluation  criteria  he  built-in  to  monitor  individual  projects 
or  the  entire  program? 

How  often  will  assessments  he  carried  out? 

What  are  the  feedback  mechanisms  within  the  program? 

Can  project  participants  assist  by  providing  feedback? 

Is  there  provision  for  periodic  evaluation  leading  to  renewal  or 
termination? 


RAND 


III.  NATIONAL  BENEFITS 


National  governments  decide  to  participate  in  formal  international 
collaborative  projects  for  a  number  of  scientific  and  political  reasons. 
Distributed  projects  may  be  easier  to  commit  to,  on  one  hand,  because 
they  do  not  require  the  same  up-front  investment  as  a  megascience 
project.  However,  they  are  more  visible  than  the  projects  taking  place  at 
the  level  of  the  individual  investigator,  thus  the  raise  questions  about 
allocation  of  resources,  sharing  of  intellectual  property  and  new 
knowledge,  and  how  to  ensure  benefits  that  are  different  from  other 
types  of  research  projects. 

A  central  finding  is  that  governmental  support  for  collaboration  depends 
on  the  policy  needs,  missions,  and  motivations  of  those  governments.  In 
fact,  we  found  that  political  and  scientific  motivations  and  goals  were 
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both  important  in  these  cases;  U.S.  government  participation  provided 
political  as  well  as  scientific  benefits.  Measuring  the  benefits  of 
collaboration  goes  beyond  the  evaluation  of  whether  the  mechanics  of 
the  program  were  effective.  This  brings  us  back  into  the  political  realm 
of  decisionmaking,  which  we  touch  on  briefly  in  this  report. 

Ensuring  that  national  goals  are  met  and  missions  addressed  does  not 
mean  that  research  has  become  “politicized,”  our  sources  told  us. 

Rather,  government  support  for  international  scientific  collaboration  was 
most  successful  when  it  was  responsive  and  accountable  to  scientific 
and  national  goals.  It  may  be  important  for  policymakers  who  consider 
initiating  or  participating  in  a  formal  collaborative  project  to  be  capable 
of  articulating  the  political  (policy)  benefits  that  will  accrue  from 
participation,  in  addition  to  highlighting  the  scientific  benefits. 
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National  Benefits  from  a  Policy 
Perspective  include  Good-will 

B  Science  &  Techndoav  Pdicv  I  nslitute _ 


From  a  policy  perspective,  U.S.  parties  said  that  the 
nation  gained: 

good  will  as  an  important  and  cooperative  partner 
credibility  for  U.  S.  policymakers  and  for  U.  S.  -funded  science 
leverage  of  investment  of  U.  S.  dollars 
accessing  infrastructure 

action  on  research  that  may  not  have  been  funded  or 
conducted  otherwise 

broader  coordinate  of  decision-making  about  science 
other  benefits  ?  Some  become  apparent  over  time. . . 


RAND 


There  are  a  number  of  benefits  that  accrue  to  the  United  States  as  a 
result  of  collaboration:  the  benefits  are  interrelated  and  overlapping.  At 
the  political  level,  the  United  States  gained  good  will  in  that  it  was  seen 
as  a  reliable  partner  by  participating  in  these  activities.  This  can  help  in 
the  future  when  the  U.S.  government  seeks  to  initiate  or  participate  in 
other  ventures.  While  it  was  initially  difficult  for  agencies  to  work  out 
how  to  participate,  they  also  gained  connections  and  political  capital  by 
being  attached  to  successful  ventures.  This  helped  U.S.  government 
officials  when  working  with  their  counterparts  in  foreign  countries.  For 
example,  participation  in  the  IPCC  helped  give  credibility  to  U.S. 
negotiators  working  on  international  climate  change  negotiations. 

Moreover,  in  the  cases  we  examined,  federal  government  agencies 
leveraged  funding:  For  the  amount  that  agencies  put  in,  participants 
estimated  that  the  United  States  leveraged  at  least  as  much  investment 
from  other  sources.  In  cases  where  U.S.  industry  participated,  federal 
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government  agencies  also  leveraged  funds  by  pooling  them  with 
industry  funds.  In  some  cases,  such  as  IMS,  industry  funds  paid  for  the 
research^,  and  U.S.  scientists  were  able  to  benefit.  In  addition,  U.S. 
scientists  gained  access  to  top  scientific  talent,  data,  and  research 
facilities  in  other  countries. 


^  The  U.S.  government  did  not  provide  funds  directly  to  IMS,  although  they  did  fund  several  projects 
through  NSF.  Other  governments  did  provide  funds,  however. 
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National  Benefits  from  a  Scientific 
Perspective  include  Access  to  Key  Resources 

H  SdenoeA  Technology Policv I nsUtuie _ 

From  a  scientific  perspective,  the  U.S.  parties  said 
that  the  nation  gained: 

access  to  key  data  sources  not  otherwise  available 

connection  to  top  foreign  scientific  talent  and  equipment 

access  to  unique  resources,  laboratories 

additional  brain  power 

new  perspectives  on  science 

greater  standardization  of  the  scientific  process 


RAND 


Based  on  the  comments  we  received  as  well  as  published  reports,  many 
scientific  benefits  also  resulted  from  these  collaborations.  In  some  of  the  cases 
we  examined,  data  was  shared  under  the  collaborative  venture  that  would  have 
been  difficult  to  share  otherwise.  International  databases  have  been  created 
that  simply  would  not  have  existed  without  collaboration.  Such  is  the  case 
with  the  genome  project.  Participation  in  collaboration  can  bring  together 
scientists  who  would  have  found  it  difficult  to  work  together  without  a  specific 
program,  enhancing  the  field  as  a  whole.  As  one  scientist  told  us,  “. .  .IPCC 
definitely  brought  scientists  together  who  I  am  sure  would  not  have  done  so 
otherwise.”  Moreover,  the  legal  and  organizational  structure  offered  by  a 
program  can  encourage  R&  that  might  not  have  taken  place:  The  IMS  terms  of 
reference,  for  example,  particularly  sections  dealing  with  intellectual  property 
rights,  have  allowed  U.S.  companies  to  protect  their  proprietary  information 
even  while  their  scientists  engage  in  meaningful  and  substantive  cooperation. 

In  several  cases,  top  scientists  who  had  received  funding  from  a  collaborative 
project  came  to  conduct  research  in  the  United  States.  This  was  the  case  with 
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HFSP,  where  many  scientists  spent  time  working  in  U.S.  labs  on 
neurosciences  research,  enhancing  the  productivity  of  U.S.  scientists. 
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Final  Observations 


•  Distributed  research  is  an  effective  form  of  coiiaboration 

•  Programs  demonstrate  new  organizationai  structure 

•  The  practices  uncovered  can  be  a  “tooikit”  for  others 

•  Evaiuation  must  account  for  this  form  of  coiiaboration 

•  Comparing  RAND’s  findings  to  others  wiii  be  cruciai 


RAND 


In  conclusion,  we  found  that  distributed  researeh  projects  can  be  an 
effeetive  way  to  facilitate  international  collaboration.  Within  the  overall 
class  of  programs  that  might  be  ealled  distributed  international 
collaboration,  different  ways  of  managing  them  will  become  apparent 
based  on  the  goals  of  the  participants,  the  nature  of  the  science  being 
pursued,  and  the  amount  of  resourees  available.  The  tools  to 
accomplish  this  organization  are  different  from  the  ones  used  for 
megascience  projeets  or  for  individual  research  efforts.  Beeause  they 
are  different,  these  programs  also  require  evaluation  tools  that  reflect 
their  unique  features  and  benefits  to  stakeholders.  Further  policy 
research  comparing  all  these  charaeteristics  across  countries  involved 
and  analyzing  various  national  and  disciplinary  approaches  will  help  to 
get  a  better  sense  of  how  to  better  manage  these  programs  in  the  future. 
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